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ABSTRACT: 

A packet transfer control apparatus, and a scheduling method therefor, having a multi-queue 
configuration provided with a plurality of queues for storing packets, wherein, when processing to 
select and transfer the head packet having the earliest scheduled transfer finish time from the 
head packets in N number of queues Qi to QN provided in correspondence with the respective 
flows, a scheduler clusters head packets having a low dependency upon changes in relative 
order of the scheduled transfer finish times with respect to changes in the entire reserved 
bandwidths of the backlog queues, holds the first order packets having the earliest scheduled 
transfer finish times among the clusters C1 to CM, reuses the output priorities in the clusters 
calculated in the past as they are at the output for transfer of the packets to calculate the 
scheduled transfer finish times for only the first order packets in the clusters among the clusters, 
and selects the packet with the earliest calculated time as the top priority packet 



Best Available Copy 



PllllllillPllWlll 

01) EP 0 989 770 A1 



(12) EUROPEAN PATENT APPLICATION 

(43) Date of publication: (51) Int CI7: H04Q 11/04, H04L 12/56, 

29.03.2000 Bulletin 2000/13 H 04 L 29/06 

(21) Application number: 99118019.1 



(22) Date of filing: 21.09.1999 



(84) Designated Contracting States: 


(72) Inventors: 


AT BE CH CY DE DK ES Fl FR GB GR IE IT LI LU 


• Xu, Yan 


MC NL PT SE 


Nakahara-ku, Kawasaki-shi, Kanagawa 211 (JP) 


Designated Extension States: 


• Chugo, Akira 


AL LT LV MK RO SI 


Nakahara-ku, Kawasaki-shi, Kanagawa 211 (JP) 


(30) Priority: 22.09.1998 JP 26767398 


(74) Representative: HOFFMANN - EITLE 




Patent- und Rechtsanwalte 


(71) Applicant: FUJITSU LIMITED 


Arabellastrasse 4 


Kawasaki-shi, Kanagawa 211 -6588 (JP) 


81925 Munchen (DE) 



(54) Packet transfer control apparatus and scheduling method therefor 




Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 



(57) A packet transfer control apparatus, and a 
scheduling method therefor, having a multi-queue con- 
figuration provided with a plurality of queues for storing 
packets, wherein, when processing to select and trans- 
fer the head packet having the earliest scheduled trans- 
fer finish time from the head packets in N number of 
queues Qj to Q N provided in correspondence with the 
respective flows, a scheduler clusters head packets 
having a low dependency upon changes in relative order 



of the scheduled transfer finish times with respect to 
changes in the entire reserved bandwidths of the back- 
log queues, holds the first order packets having the ear- 
liest scheduled transfer finish times among the clusters 
C 1 to C M , reuses the output priorities in the clusters cal- 
culated in the past as they are at the output for transfer 
of the packets to calculate the scheduled transfer finish 
times for only the first order packets in the clusters 
among the clusters, and selects the packet with the ear- 
liest calculated time as the top priority packet. 
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Description 

BACKGROUND OF THE INVENTION 
5 1 . Field of the Invention 

[0001] The present invention relates to a packet transfer control apparatus having a multi-queue configuration pro- 
vided with a plurality of queues for storing packets and a scheduling method therefor, more particularly relates to a 
packet transfer control apparatus for efficiently selecting and outputting a packet capable of finishing being transferred 

io the earliest from among head packets in the queues and a scheduling method therefor. 

[0002] In a packet transfer network, in order to guarantee the quality of service (QoS) for every flow (a plurality of 
consecutive packets which a user has requested the transfer of) and to utilize resources such as the transmission 
paths (bandwidth available for transmission) to the maximum to improve the service, it is requested that the packet 
transfer control apparatus be provided with a "weighted multi-queueing" functbn where it is provided with a plurality 

is of queues corresponding to the flows and weight the queues with an order of transfer so as to control them for a fair 
packet transfer among queues and a 'fair queueing* function where remaining resources (leftover bandwidth) are fairly 
distributed among flows temporarily so as to make efficient use of the resources. 

[0003] In the case of traffic data such as file transfer protocol (FTP) data or world wide web (WWW) data, it is con- 
sidered important to transfer the data of each flow fairly at a high speed by utilizing the resources to the maximum limit. 
20 The routers etc. of gateways of the network have been required to perform high speed processing, in correspondence 
with the high speed output links, of control for allocating to each flow and guaranteeing a bandwidth reserved for each 
of these flows (or sets of flows) and, at the same time, fairly reallocating the leftover resources in output links among 
these flows. 

[0004] On the other hand, in the case of voice traffic data such as voice over IP (VoIP), it is more important that a 
2S fixed transfer rate be guaranteed and a delay jitter be minimized rather than the bandwidth of the links being used fairly 
and to the maximum limit. 

[0005] The present invention deals with the transfer of packets with different required qualities of service (QoS), 
focuses on: 

30 1 ) a structure for performing high speed processing of transfer order control (scheduling) of individual packets in 

accordance with different requests for quality of service for every type of the traffic data and 
2) a structure for linking these individual structures and effectively utilizing the entire bandwidth, and 
will be applied to high speed multimedia networks in the future. 

3S 2. Description of the Related Art 

[0006] As the fair queueing technique, weighted fair queueing (WFQ) and start-time fair queueing (SFQ) are well 
known. 

[0007] The principle of fair queueing based on this weighted fair queueing (WFQ) will be explained in detail later by 
40 using Fig. 5. As clear from the explanation using Fig. 5, in transfer control by weighted fair queueing (WFQ), the 
scheduled transfer finish times Fj are calculated for the respective head packets in the queues Qj (i = 1, 2, .... N) and 
priority is assigned to the order of output by the calculated value so as to achieve fairness of the packet transfer among 
flows and the effective utilization of resources. 

[0008] The scheduled transfer finish times Fj tl of the head packets in the queues Qj are calculated by equation (1) 
45 given later. 

[0009] The calculation of equation (1 ) includes the calculation of the bandwidths allocated to the queues Q. These 
vary due to the change of the backlog queues with every transfer of a packet as shown in equation (2) and equation 
(3) given later. The allocated bandwidths r 8 ■, of the next head packets cannot be correctly calculated before the end of 
the output processing of the packets immediately before them. 
so [0010] Namely, whenever a new head packet is output, the calculation must be redone. In addition, the results of 
the calculations for the scheduled head packet transfer finish times F { , of the queues calculated at the time of the 
previous packet outputs cannot be used at all for the next packet outputs even for head packets remaining in the 
queues Qj without being selected. This invited an increase in the amount of calculation. 

[0011] On the other hand, known in the art is the technique of introducing virtual time in order to solve the problem 
55 of the increase of the amount of calculation due to the change of the allocated bandwidths r- 1 . By utilizing virtual time, 
equation (1) can also be represented by an equation including a virtual time V(t), a sum value ^ of the reserved 
bandwidths of the backlog queues and the resource bandwidth R as will be explained later. 

[0012] The virtual time V(t) is recalculated whenever the sum value <I>b of the reserved bandwidths of the backlog 
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queues and the resource bandwidth R change, but realization is easy since the scheduled transfer finish time F 0 can 
be determined at the time of arrival of the packet. 

[0013] When virtual time is introduced, the time axis of the scheduled transfer finish time F S1 ends up becoming its 
own axis which is different from the actual time axis. Thus, when a plurality of schedulers for controlling the order of 

5 transfer of the packets are provided in accordance with the quality of service request (QoS), the time axes end up 
becoming different among schedulers. Therefore, there is a problem that the link among schedulers, for example, the 
processing for selecting the output packet having the top priority among the schedulers, becomes difficult. 
[0014] Further, it has been sometimes pointed out that, if a control of the packet transfer is achieved by the fair 
reallocation of resources like weighted fair queueing (WFQ) without using virtual time, the amount of calculation for 

10 determining the order of the transfer (scheduling) becomes extremely large, so it is difficult to apply this to a high speed 
link. 

SUMMARY OF THE INVENTION 

15 [001 5] Therefore, an object of the present invention is to provide a packet transfer control apparatus, and a scheduling 
method therefor, which greatly reduce the amount of calculation for scheduling of the packet transfer in fair queueing 
for guaranteeing different qualities of service (QoS) for the respective flows and utilizing the resources of the trans- 
mission paths to the maximum limit, which enable high speed packet transfer processing, and which in addition can 
easily achieve a link among a plurality of provided schedulers in accordance with the requested qualities of service 

20 (QoS). 

[0016] When further concretely describing this, to solve the problem, in the calculation for selecting the top priority 
packet to be output for transfer, (i) the range of the packets directly influenced by the change of the bandwidths r x f1 
allocated to the queues Qj is limited so as to calculate the scheduled transfer finish times F s , with respect to only limited 
packets and (ii) the results of the past calculations are reused for other packets to omit the above calculations, whereby 

25 a reduction of the amount of calculation of the weighted fair queueing (WFQ) is achieved. 

[0017] To achieve the above object, the present invention provides a packet transfer control apparatus having a 
multi-queue configuration provided with a plurality of queues each storing packets and a scheduling method therefor 
wherein, when processing to select and transfer the head packet having the earliest scheduled transfer finish time from 
the head packets in N number of queues Qj to Q N (1-2) provided in correspondence with the respective flows, a sched- 

30 uler (1-1) clusters head packets having a tow dependency upon changes in relative order of the scheduled transfer 
finish times with respect to changes in the entire reserved bandwidths of the backlog queues, holds the first order 
packets having the earliest scheduled transfer finish times among the clusters G, to C M , reuses the output priorities 
in the clusters calculated in the past as they are at the output for transfer of the packets to calculate the scheduled 
transfer finish times for only the first order packets in the clusters among the clusters, and selects the packet with the 

35 earliest calculated time as the top priority packet. As a result, the amount of calculation for selecting the packet to be 
transferred with the top priority is greatly reduced whereby high speed packet transfer processing is enabled. 

BRIEF DESCRIPTION OF THE DRAWINGS 

40 [0018] The above object and features of the present invention will be more apparent from the following description 
of the preferred embodiments given with reference to the accompanying drawings, wherein: 

Fig. 1 is an explanatory view of the principle of packet transfer control of the present invention; 
Fig. 2 is a graph of the relationship between a scheduled transfer finish time F if1 and a reserved bandwidth use ratio; 
45 Figs. 3A and 3B are views of the configuration of a packet transfer control apparatus of the present invention; 

Fig. 4 is an explanatory view of the packet transfer control by a plurality of schedulers in the present invention; and 
Fig. 5 is an explanatory view of the principle of weighted fair queueing (WFQ). 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

so 

[0019] Before describing the embodiments of the present invention, the related art and the disadvantages therein 
will be described with reference to the related figures. 

[0020] As already explained, as fair queueing techniques, weighted fair queueing (WFQ) and start-time fair queueing 
(SFQ) are known. 

5S [0021] The principle of the fair queueing based on weighted fair queueing (WFQ) will be explained by using Fig. 5. 
In Fig. 5, 5-1 is a scheduler for packet transfer, 5-2 are meters provided in correspondence with the queues, and 5-3 
is a multi-queue comprised by N number of queues (Qj to Q N ) corresponding to the flows. 

[0022] Each queue 5-3 stores packets for every flow. The packets are processed with FIFO in each queue. The 
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scheduler 5-1 performs the processing for selecting the packet for which the transfer can be finished earliest from 

among the head packets stored in the queues 5-3 and outputting this packet to the output link. 

[0023] In this way, in transfer control by weighted fair queueing (WFQ), the scheduled transfer finish times F lA are 

calculated for the head packets of all of the queues Q (i = 1 , 2 N) and priority is assigned to the order of output by 

the thus calculated values to achieve fairness of the packet transfer among flows and effective utilization of resources. 
[0024] The scheduled transfer finish times F lfl of the head packets in the queues Qj are calculated by the following 
equation (1): 

F i.i = ""Pm + L i.o\o; Hi) + kAi ( 1 ) 



[0025] Here, max { ; } means the maximum value between the two values divided by the semicolon in the brackets. 
Further, Sj 0 etc. are as follows. Note that the first subscript is the number of the queue (flow), and the second subscript 
is the order in the queue. 

75 [0026] S i0 is the transfer start time of the packet immediately preceding the present head packet in a queue 
(hereinafter referred to as the "preceding packet"), 

L j 0 is the packet length of the preceding packet in a queue Qj, 

rj o is the bandwidth allocated to a queue Qj at the time of transfer of the preceding packet in the queue Qj, 
20 tj \ is the time when the head packet in a queue Qj arrived at (is stored in) the queue Qj, 

L- 1 is the packet length of the head packet in a queue Qj, and 

^ i is the bandwidth allocated to a queue Qj at the time of transfer of the head packet in the queue Qj. 

[0027] Sj 0 , Lj 0 , L i1t and t| §1 mentioned above are fixed values determined at the start of the packet transfer or the 
25 arrival of the packet. The values thereof are stored and held in the meters 5-2 in correspondence with the queues C^, 
but fj o and rj 1 mentioned above must be obtained by calculation as follows: 



R i0 = (O/0> b(0) ) X R (2) 

[0028] Here, <Dj is the reserved bandwidth of a queue Qj, i.e., the bandwidth allocated for the packet transfer to the 
queue Qj. 

[0029] O b ( 0 ) is the sum value of the reserved bandwidths for the backlog queues at the time of transfer of the preceding 
packet, that is, O b(0) = X<&) (j: backlog queues at the time of transfer of the preceding packet), 

0> b(1) is the sum value of the reserved bandwidths for the backlog queues at the time of transfer of the head 
packet, that is, O b(1 ) = X<D) (j: backlog queues at the time of transfer of the head packet), and 

R is the entire output bandwidth of the resources, where < R and ^(i) ^ R stand. 
[0030] Further, the above mentioned "backlog queue" means a queue in which at least one u transferred packet 
(backlog) is stored and is a queue in which at least one packet requiring a transfer operation is stored, therefore is also 
referred to as an "active queue". 

[0031] The bandwidth rj allocated to each queue Q { varies according to the number of the backlog queues. As ap- 
parent from equation (2) or (3), when 0^ = R, it becomes the bandwidth Oj, while when <D b = 0> i( it becomes the maximum 
bandwidth R. 

[0032] Here, the first term on the right side of equation (1 ) represents the transfer start reference time of the head 
packet in a queue Qj. The larger value between the transfer finish time (Sj 0 + Lj,(/r it0 ) of the preceding packet and the 
head packet arrival time t| tl is given as the transfer start reference time. The second term on the right side of equation 
(1 ) is the time required for the transfer of the head packet. 

[0033] The scheduler 5-1 performs control for selecting and transferring the packets in the order from the one having 
the smallest value of the scheduled transfer finish time Fj fl . Accordingly, the priority order of transfer is lowered for a 
queue requiring a long time for the transfer of the preceding packet and a queue with a head packet requiring a long 
time for the transfer, thus fair packet transfer is carried out among queues. 

[0034] In this way, in weighted fair queueing (WFQ), the weighted control of the order of transfer by the scheduled 
transfer finish times F f t is carried out with respect to the packets for all of the queues to achieve fair packet transfer 
among queues and, at the same time, unused reserved bandwidths (remaining resources) are fairly allocated to active 
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queues in accordance with their reserved bandwidths to effectively reuse the resources, and thereby the packets in 
active queues are transferred at a higher speed so as to improve the service. 

[0035] However, the number of backlog queues always varies, so the bandwidths r { to be allocated to the queues 
Qi always vary as well and therefore the scheduled transfer finish times F S1 of the head packets in the queues Q vary. 
[0036] Accordingly, the scheduler 5*1 had performed the processing for calculating the scheduled transfer finish 
times Fj 1 of the head packets by equation (1 ), equation (2), and equation (3) for all of the head packets in the backlog 
queues whenever one packet was output and then selecting, transferring, and outputting the head packet in the queue 
Q ou t witn tne smallest scheduled transfer finish time F^ j from among these calculated values as described in the 
following equation (4): 



F out,i = min { F i,i . » : backlog queue} (4) 

[0037] The calculation of equation (1 ) mentioned above includes the calculation of the bandwidths r s 0 and allo- 
15 cated to the queues Qj. These vary due to the change of the backlog queues with every transfer of a packet as shown 
in equation (2) and equation (3). The allocated bandwidths of the next head packets cannot be correctly calculated 
before the end of the output processing of the packets immediately before this. 

[0038] Namely, whenever a new head packet is output, the calculation must be redone. In addition, the results of 
the calculations for the scheduled head packet transfer finish times Fj f1 of the queues calculated at the time of the 
20 previous packet outputs cannot be used at all for the next packet outputs even for head packets remaining in the 
queues Q } without being selected. This invited an increase in the amount of calculation. 

[0039] On the other hand, known in the art is the technique of introducing virtual time in order to solve the problem 
of the increase of the amount of calculation due to the change of the allocated bandwidths r it1 . By utilizing virtual time, 
equation (1) can be interpreted as: 

25 

Fj^maxlFj^V^H + L^/O^ (V) 

where, 

V(t ii1 ) = V(t i0 ) + (FViI> b ).(t il -t i0 ) 

V(0) = 0 

35 [0040] The virtual time V(t) is recalculated whenever the sum value O b of the reserved bandwidths of the backlog 
queues and the resource bandwidth R change, but realization is easy since the scheduled transfer finish time Fj j can 
be determined at the time of arrival of the packet. 

[0041] However, when virtual time is introduced, the time axis of the scheduled transfer finish time F {1 ends up 
becoming unique its own axis which is different from the actual time axis. Thus, when a plurality of schedulers for 
40 controlling the order of transfer of the packets are provided in accordance with the quality of service request (QoS), 
the time axes end up becoming different among schedulers. Therefore, there is a problem that the link among sched- 
u lers, for example, the processing for selecting the output packet having the top priority among the schedulers, becomes 
difficult. 

[0042] Further, it has been sometimes pointed out that, if a control of the packet transfer is achieved by the fair 
45 reallocation of resources like weighted fair queueing (WFQ) without using virtual time, the amount of calculation for 
determining the order of the transfer (scheduling) becomes extremely large, so it is difficult to apply this to a high speed 
link. 

[0043] The present invention provides a packet transfer control apparatus, and a scheduling method therefor, which 
greatly reduce the amount of calculation for scheduling of the packet transfer in fair queueing for guaranteeing different 
qualities of service (QoS) for the respective flows and utilizing the resources of the transmission paths to the maximum 
limit, which enable high speed packet transfer processing, and which in addition can easily achieve a link among a 
plurality of provided schedulers in accordance with the requested qualities of service (QoS). 

[0044] When further concretely describing this, to solve the problem, in the calculation for selecting the top priority 
packet to be output for transfer, (i) the range of the packets directly influenced by the change of the bandwidths r itl 
allocated to the queues Q, is limited so as to calculate the scheduled transfer finish times F s -, with respect to only limited 
packets and (ii) the results of the past calculations are reused for other packets to omit the above calculations, whereby 
a reduction of the amount of calculation of the weighted fair queueing (WFQ) is achieved. 



5 



EP 0 989 770 A1 

[0045] In general, the present invention provides a packet transfer control apparatus comprising: a scheduler for 
controlling the transfer by weighting the order of transfer for each head packet in the queues, the scheduler comprising: 
means for dividing each head packet in the queues into clusters in accordance with the ratio between the packet length 
of the head packet in each queue and the reserved bandwidth of the packet, and means for selecting the packet having 
s the earliest scheduled transfer finish time as the top priority transfer packet from among each first order packet inside 
the clusters. 

[0046] According to the present invention, (1 ) there is provided a packet transfer control apparatus provided with at 
least one scheduler which controls the transfer by weighting the order of transfer for the head packets in the queues 
in the order from the packet having the earliest scheduled transfer finish time for packets stored in a plurality of queues 

10 to which are allocated reserved bandwidths individually determined in advance and, at the same time, allocating the 
resources (bandwidths) of the entire transmission system to all of the active queues in which packets to be transferred 
are stored, wherein the scheduler is provided with a clustering means for dividing the head packets in the queues into 
clusters in accordance with the ratio between the packet length of the head packet in each queue and the reserved 
bandwidth (hereinafter referred to as the scheduled resource occupation time), a cluster managing means for holding 

is the packets having the earliest scheduled transfer finish times among the packets in a cluster for all clusters as the 
first order packets, an inside-cluster selecting means for selecting the packet having the earliest scheduled transfer 
finish time for the cluster as the first order packet when a packet is deleted or added inside the cluster, and an inter- 
cluster selecting means for selecting the packet having the earliest scheduled transfer finish time as the top priority 
transfer packet from among the first order packets in the clusters. 

20 

(2) The scheduler may be provided with a reserved bandwidth use ratio computing means for computing the ratio 
(hereinafter referred to as the reserved bandwidth use ratio) of a cumulative value of the reserved bandwidths for 
the active queues with respect to the resources (bandwidths) of the entire transmission system, and said inter- 
cluster selecting means is structured to calculate the scheduled transfer finish time based on the transfer start 

25 reference time of each first order packet, the scheduled resource occupation time, and the reserved bandwidth 

use ratio. 

(3) The inside-cluster selecting means may further be structured to sort the packets in a cluster in the order from 
the earliest scheduled transfer finish time, and the cluster managing means may be constituted so as to hold the 
packets sorted in the order from the earliest scheduled transfer finish time for every cluster. 

30 (4) The inside-cluster selecting means may be structured to perform the selection by switching between processing 

for selecting only the first order packet having the earliest scheduled transfer finish time from the packets in a 
cluster and processing for sorting them in the order from the one having the earliest scheduled transfer finish time, 
perform the processing for sorting in the order from the one having the earliest scheduled transfer finish time when 
the transfer time of a packet in transmission is more than a predetermined time, and perform the processing for 

35 selecting only the first order packet having the earliest scheduled transfer finish time when the transfer time of the 

packet in transmission is not more than a predetermined time. 

(5) The inside-cluster selecting means may further be constituted so as to select the packet having the earliest 
scheduled transfer finish time based on only the transfer start reference time of each packet in the cluster or the 
transfer start reference time and the scheduled resource occupation time. 
40 (6) The inside-cluster selecting means may further be constituted so as to sort the packets in the cluster in the 

order from the one having the earliest scheduled transfer finish time based on only the transfer start reference 
time of each packet in the cluster or the transfer start reference time and the scheduled resource occupation time. 

(7) The clustering means may be constituted so as to divide the head packets in the queues into clusters by 
discriminating the type of the application packet having traffic characteristics for which the scheduled resource 

45 occupation time is specified. 

(8) The packet transfer control apparatus may be provided with a plurality of schedulers which perform different 
processing for allocation of the resources of the entire transmission system for packets stored in multi-queues 
differing for all clusters in accordance with the class of the quality of service, in this case, each scheduler is provided 
with a means for selecting the packet having the earliest scheduled transfer finish time from among the head 

so packets for every multi-queue. Further, provision is made of a scheduler for selecting the packet having the earliest 

scheduled transfer finish time from among packets selected by the schedulers corresponding to the multi-queues. 

(9) At least one scheduler among the schedulers corresponding to the multi-queues is provided with a means for 
allocating bandwidths of a fixed rate to queues in the corresponding multi-queues and selecting the packet having 
the earliest scheduled transfer finish time. 

55 

[0047] According to another aspect of the present invention, (10) there is provided a packet transfer scheduling 
method which transfers packets by weighting the order of transfer for head packets of queues in the order from the 
packet having the earliest scheduled transfer finish time for packets stored in a plurality of queues to which are allocated 
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at least reserved bandwidths individually determined in advance and, at the same time, allocating resources (band- 
widths) of the entire transmission system to all of the active queues in which packets to be transferred are stored, 
comprising a step of dividing the head packets of the queues into clusters in accordance with scheduled resource 
occupation times of the head packets in the queues, a step of selecting the packet having the earliest scheduled transfer 
5 finish time for a cluster as a first order packet when a packet is deleted or added to packets in the cluster and holding 
this selected packet, and a step of selecting the packet having the earliest scheduled transfer finish time from among 
the first order packets in the clusters as a top priority transfer packet. 

(11 ) The packet transfer scheduling method may further comprise a step of computing a reserved bandwidth use 
10 ratio of active queues and a step of calculating a scheduled transfer finish time during the selection among clusters 

based on the transfer start reference time of the first order packet, the scheduled resource occupation time, and 
the reserved bandwidth use ratio. 

(12) The packet transfer scheduling method may further comprise, when achieving the selection in the clusters, a 
step of sorting the packets in a cluster in the order from the earliest scheduled transfer finish time and holding the 

15 sorted packets. 

(13) The packet transfer scheduling method may further comprise, when achieving the selection in the clusters, a 
step of performing processing for sorting in the order from the packet having the earliest scheduled transfer finish 
time when the transfer time of a packet in transmission is more than a predetermined time and performing process- 
ing for selecting only the first order packet having the earliest scheduled transfer finish time when the transfer time 

20 of the packet in transmission is not more than a predetermined time. 

(14) The packet transfer scheduling method may further comprise, when achieving the selection in the clusters, a 
step of selecting the packet having the earliest scheduled transfer finish time based on only the transfer start 
reference time of each packet in the cluster or the transfer start reference time and the scheduled resource occu- 
pation time. 

25 (15) The packet transfer scheduling method may further comprise, when achieving the selection in the clusters, a 

step of sorting the packets in the cluster in the order from the one having the earliest scheduled transfer finish time 
based on only the transfer start reference time of each packet in the cluster or the transfer start reference time 
and the scheduled resource occupation time. 

(16) The packet transfer scheduling method may further comprise, when achieving the selection in the clusters, a 
30 step of dividing the head packets of queues into clusters by discriminating the type of the application packet having 

traffic characteristics for which the scheduled resource occupation time is specified. 

[0048] Turning now to specific embodiments of the present invention, Fig. 1 is an explanatory view of the principle 
of the packet transfer control by weighted fair queueing (WFQ) according to the present invention. In the figure, 1-1 is 
35 the scheduler for controlling transfer by clustering the packets, and 1 -2 is the multi-queue comprising N number of 
queues (Qj to Q N ) for storing packets in correspondence with flows. 

[0049] The internal structure of the queues Qj (i = 1 , 2, ••«, N) does not have any direct bearing upon the packet 
transfer control of the present invention, so it is assumed that the queues Qj are comprised so as to perform the packet 
storing processing by the general FIFO (First in First Out) method. 

40 [0050] The scheduler 1 -1 performs the clustering of the head packets in the queues Q} (hatching part (A) in the figure) 
based on the scheduled resource occupation time ft ^ required for the transfer of each packet. Figure 1 shows an 
example of controlling the transfer by classifying the head packets in the queues Qj into M number of clusters C-, to C M . 
[0051] The scheduled resource occupation time ft f1 required for each packet transfer is calculated from the packet 
length of the head packet in each queue Qj and the reserved bandwidth <£; of each queue Qj by the following 

45 equation: 

Pi.i= L i.i'*i 

so [0052] Namely, p M is the time required for transferring the packet having the packet length Lj A under the reserved 
bandwidth <Dj minimally allocated to the queue Qj. 

[0053] In actuality, as mentioned above, due to the weighted fair queueing (WFQ), if there is unused reserved band- 
width, that unused reserved bandwidth is also allocated to the active queues with a predetermined ratio. Accordingly, 
the resource occupation time will become shorter than the ft 1t so the ft t1 will be referred to as the scheduled resource 
55 occupation time. 

[0054] The scheduler 1 -1 clusters each head packets into any one of M number of clusters to C M in accordance 
with the values of the scheduled resource occupation times ft of the head packets. Figure 1 shows an example of a 
case where a head packet P 5 in the queue Q 5 and a head packet P 3 tl in the queue Q 3 are clustered into the cluster 
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C 1( a head packet P 21 in the queue Q 2 and a head packet P 4 , in the queue Q4 are clustered into the cluster C 2 , and 
a head packet P, (1 in the queue is clustered into the cluster C M . 

[0055] The scheduler 1-1 registers the head packet having the scheduled transfer finish time F itl of the smallest 
value in the cluster as the first order packet to be output with the top priority in this cluster when newly incorporating 
s a head packet in a queue into the cluster. 

[0056] Further, when the first order packet in a cluster is selected as the transfer packet and sent to the output link, 
it also registers the packet having the scheduled transfer finish time F itl of the smallest value from among the remaining 
packets in the cluster as the first order packet in the cluster. 

[0057] In Fig. 1 , the example where the packet P 2 A is sent from the cluster C 2 and the next packet P 2 ^ is incorporated 
10 from the queue Q 2 into the same cluster C 2 as the head packet is shown. When packets constituting one cluster change 
in this way, the packet having the smallest value of scheduled transfer finish time F lA among the packets in that cluster 
(hatching part (B) in the figure) is registered as the first order packet. 

[0058] The scheduler 1-1 performs control for calculating the scheduled transfer finish times F t , for the first order 
packets in the clusters (packets P 5 A , P 2#1 , .... Pi ,1 of the hatching part (C) in the example of Fig. 1 ), selecting the packet 
is having the smallest value of F { f1 as the packet to be transferred with the top priority and sending the selected packet 
from the output link. 

[0059] The principle of the packet transfer control of the present invention will be explained in further detail. The 
scheduler 1-1 shown in Fig. 1 is provided with meters (not shown) in correspondence with the queues q. The meters 
are set with the parameters required for calculating the scheduled transfer finish times F s tl of the head packets in the 
20 queues. 

[0060] The parameters set in the meters for the queues Qj are the transfer start reference time t of the head 
packets and the scheduled resource occupation times ft «, of the head packets. The transfer start reference time 
of a head packet is calculated by the following equation: 

25 

a- t , = max{Sj 0 + L } 0 /r iQ ; t M } (5) 

[0061] Here, S i 0 + Lj 0^ 0 is the scheduled transfer finish time of the packet immediately preceding the head packet 
(preceding packet), and t k1 is the arrival time of the head packet. The transfer start reference time of the head 
30 packet becomes the larger value between values of the scheduled transfer finish time of the preceding packet and the 
arrival time of the head packet. 

[0062] The scheduled resource occupation time ft f1 is calculated by: 

as Pi.i= L i/*i m 

as mentioned above. The scheduled resource occupation time ft A is the resource occupation time required for the 
transfer when the packets in the queues Q are transferred with the reserved bandwidth Oj as the lowest allocated 
bandwidth. The bandwidths allocated to the queues Q vary according to the change of the backlog queues, so the 
40 resource occupation times also vary. Accordingly, the scheduled resource occupation time ft A is not an actual sched- 
uled resource occupation time but one of the reference values used for the calculation of the scheduled transfer finish 
time F S1 as will be explained later. 

[0063] The scheduler 1 -1 calculates the reserved bandwidth use ratio y of the backlog queues as the parameter for 
further calculating the scheduled transfer finish times F jtl of the head packets. The reserved bandwidth use ratio y is 
45 the ratio of the sum value of the reserved bandwidths used by the backlog queues (active queues) with respect 
to the resources (bandwidth of entire output link) R and is calculated by the following equation: 

Y=<W R < 7 > 

so 

[0064] The scheduled transfer finish time of the head packet of equation (1 ) can be modified as in the following 
equation (8) using the parameters otf fl , ft ?1 , and y according to equation (3), equation (5), equation (6), and equation (7): 

55 
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Fi.i = + L ia /{(^/^biu) x R} 

= Ctx.i + Px.i * Y (8) 

[0065] Here, the transfer start reference time Oj t1 and the scheduled resource occupation time ft t1 are calculated at 
the time when a packet becomes the head of the queue and are fixed values which will not change later, but the 
10 reserved bandwidth use ratio y is a value changing according to the change of the backlog queues. 

[0066] When representing the relationship between the scheduled transfer finish times F^, and the reserved band- 
width use ratio y by equation (8) in a graph, as indicated by the slanted lines in (A) of Fig. 2, the scheduled transfer 
finish times F j(1 change linearly with respect to a change of y. 

[0067] (A) of Fig. 2 shows an example of the scheduled transfer finish times Fj (1 (i = 1 to 4) of the head packets of 
is the queues Qj (i = 1 to 4). The points of intersection between the slanted lines and the ordinate of the figure are otj v 
and the gradients of the slanted lines are ft v 

[0068] As will be understood from (A) of Fig. 2, in the case of scheduled transfer finish times Fj ., of the packets not 
having a large difference of the gradient ft t1 of the slanted line, even if the reserved bandwidth use ratio changes, the 
probability of a change in the order of the scheduled transfer finish times F 0 of these packets is low. Even if there is 

20 a change, the time difference is minute. 

[0069] In (A) of Fig. 2, there is not a large difference between the gradients ft A of F| (1 where i = 1 and Fj #1 where i = 
3 and also there is not a large difference between the gradients ft (1 of Fj j where i = 2 and Fj t1 where i = 4. In such a 
case, the relationship of magnitude between Fj j where i = 2 and F l(1 where i = 4 does not change when there is a 
change of the reserved bandwidth use ratio y Further, the relationship of magnitude between Fj ^ where i = 1 and 

25 where i = 3 inverts after the two Fj •, intersect midway, but since the gradients ft ■, are close to each other, a large 
difference does not occur between the values of F l?1 where i = 1 and F jfl where i = 3 per se. 

[0070] (B) of Fig. 2 is a view of only the gradient ft ^ . Several (for example M number of) ranges are determined in 
advance for this gradient ft v The clustering is carried out with respect to the head packets of the queues as mentioned 
before according to which range the gradient ft t1 calculated by equation (6) belongs. 
30 [0071] When performing the clustering of the head packets in this way, it is possible to make the dependency of the 
order of the scheduled transfer finish times F it1 of packets belonging into same cluster with respect to the change of y 
minute. 

[0072] For example, in Fig. 2, when performing the clustering while assuming that the head packets in the queues 
where i = 1 and i = 3 belong to the same cluster and that the head packets in the queues where i = 2 and i = 4 belong 

35 to the same cluster, the probability of change of the order of the scheduled transfer finish times F-^ of the packets in 
queues Qj in the clusters calculated when y= y, is low even when y = y 2 . Even if such a change occurs, the time 
difference thereof is small. Accordingly, even if an error occurs in the order of transfer, the influence thereof is small. 
[0073] The packet selection processing at the time of packet transfer output includes (i) processing for calculating 
the scheduled transfer finish times Fj ^ by equation (8) for all packets in a cluster which output a packet immediately 

40 before or a cluster to which a new head packet is added (hatching part (B) of Fig. 1 ) and selecting the first order packets 
in which the calculated scheduled transfer finish times Fj t becomes the minimum (inside-cluster selection processing) 
and (ii) processing for calculating the scheduled transfer finish times Fj j by equation (8) between first order packets 
in the clusters (hatching part (C) of Fig. 1 ) and selecting the packet in which the calculated value becomes the minimum 
by equation (4) (inter-cluster selection processing). 

45 [0074] Since it is sufficient to perform only the processing for sending the packet selected by the selection processing 
among clusters as the top priority transfer packet to the output link, therefore the amount of calculation can be greatly 
reduced compared with the case of the related art where the calculation of equation (8) was carried out with respect 
to the head packets in all backlog queues. 

[0075] For example, if there are 100 backlog queues, there are 100 head packets for these backlog queues. In 
50 contrast to the related art where the processing for calculating the scheduled transfer finish times Fj by equation (8) 
for 100 packets and selecting the packet having the minimum calculated value, in the present invention, when assuming 
that 100 head packets are clustered into 10 clusters each comprising 10 head packets on an average, it is sufficient 
to perform the inside-cluster selection processing for an average 10 packets and the inter-cluster selection processing 
for 10 clusters. Namely, it is sufficient to calculate the scheduled transfer finish times F l} by equation (8) for only 20 
55 packets, so the amount of calculation can be greatly reduced. At the same time, in the processing for selecting the 
one having the minimum calculated value as well, the number of objects to be selected is greatly decreased, so the 
amount of processing in this selection processing can be lowered. 

[0076] In general, where the number of queues is N and the number of clusters is M, the number of the packets in 
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the cluster becomes N/M on an average. Namely, by performing the clustering as in the present invention, the calculation 
which had been carried out with respect to N number packets in the related art may be carried out by only the inside- 
cluster processing with respect to N/M number packets and the inter-cluster processing with respect to M number ot 
packets. Accordingly, since It is sufficient to carry out the calculation with respect to (N/M + M) number of packets, the 

5 amount of calculation can be greatly reduced. 

[0077] Note that, in the inside-cluster selection processing mentioned above, in place of the processing for selecting 
the first order packets by the calculation of the scheduled transfer finish times F i A by equation (8) using otj #1 , ft v and 
y, the following simple processing is also possible. In a cluster, the scheduled resource occupation times ft A of packets 
are values close to each other and the dependency upon the reserved bandwidth use ratio y is low, therefore packets 

10 are sorted in the order from the packet having the smallest value of the transfer start reference time 04 (1 as the first 
sort key. Next, the packets having transfer start reference times Oj of values which are the same as or close to each 
other are sorted in the order from the packet having the smallest scheduled resource occupation times ft ^ as the 
second sort key. Due to this, the packet having the scheduled transfer finish time Fj near the smallest value can be 
selected by simple processing without using the reserved bandwidth use ratio y. 

is [0078] Further, by using only the value of the transfer start reference time otj t , even if the packet having the smallest 
value is selected as the first order packet, a large error will not occur in the selection result and the packet having the 
scheduled transfer finish time Fj -, near the smallest value can be selected by further simple processing. In this case, 
the calculation of p x y is not carried out, therefore the amount of calculation for the inside-cluster processing can be 
greatly reduced. 

20 [0079] Further, in the inside-cluster selection processing, when performing selection using otj ^ , ft 1f and y mentioned 
above or selection using only Oj tl and ft -,, it is also possible to apply (i) a procedure of registering only the selected 
first order packet, comparing the magnitude of the scheduled transfer finish times F ifl between the already registered 
first order packet and the packet to which a new packet is added when a new packet is added to the cluster, determining 
a new first order packet, and registering this, (ii) a procedure of sorting all packets in the cluster in the order from the 

25 packet having the smallest scheduled transfer finish time F^, and further (iii) a procedure of switching the above 
procedures in accordance with the processing time permitted to the inside-cluster selection processing. 
[0080] In general, there is a variation in lengths of packets. By utilizing this characteristic, when packets requiring a 
long time for transfer such as long packets are sent, a targe time is given for performing the calculation for selecting 
the next output packet, therefore the packets in clusters are sorted during that time. When sending short packets at 

30 high speed or in selection processing in a cluster in which there has only been deletion, but no addition of a packet, 
by using the past sorted results as they are or by switching the operation to the processing for selecting only the first 
order packets, the amount of processing can be reduced. Even in a case where consecutive short packets are output, 
high speed selection processing can be carried out without deterioration of the precision or while accompanied with 
only a slight reduction of precision. 

35 [0081] Note that whether or not a packet is one requiring a long time for transfer can be judged prior to the start of 
the sorting by division of the packet length L by the entire output bandwidth R, i.e., L7R, and the comparison of the 
calculated value with a threshold value. It is also possible to perform simple judgement by comparison of only the 
packet length with a threshold value. 

[0082] Further, when calculating the scheduled transfer finish times Fj j for selecting a first order packet in inside- 
40 cluster selection processing, the inter-cluster selection processing calculates the scheduled transfer finish times Fjj 
for only the first order packets of the clusters other than the cluster for which the inside-cluster selection processing is 
being carried out, compares the smallest value among these calculated values with the scheduled transfer finish time 
Fj 1 calculated in the inside-cluster selection processing, and thereby selects the top priority packet to be transferred. 
[0083] Further, in the transfer of the application packets in multimedia such as voice information or video information, 
45 in general, packets are transferred after classifying the packets into some grades in accordance with the required 
quality. 

[0084] In each above grade, the traffic characteristics such as the reserved bandwidth <J>j and the packet length Lj 
are uniform. There are cases where the scheduled link occupation time ft (ft = L/Oj) is uniquely determined by dis- 
criminating the type of the application. In such a case, by discriminating the type of the traffic characteristic of the 

so application packet, the clustering of the packets can be easily and correctly carried out. 

[0085] Next, an explanation will be made of a concrete configuration of a packet transfer control apparatus using 
weighted fair queueing (WFQ) according to the present invention. Here, it is assumed that the packet transfer control 
apparatus performs the queueing for every flow. It performs the storage processing of packets by using FIFO inside 
each queue. Further, the packet data per se is stored in a not illustrated buffer. Further, it is assumed that the packet 

55 transfer control apparatus explained below processes only pointers for the packet data per se and parameters con- 
cerning the transfer control. 

[0086] Figures 3A and 3B are views of the configuration of the packet transfer control apparatus of the present 
invention. In the figure, 3-1 is a multi-queue managing means, 3-2 is a reserved bandwidth use ratio 7 computing 
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means, 3-3 is a clustering means, 3-4 is a cluster managing means, 3-5 is an inside-cluster selecting means, 3-6 is 
an inter-cluster selecting means, and 3-7 is an output control means. 

[0087] The multi-queue managing means 3-1 is provided with queues Q-, (i = 1 to N) in correspondence with flows 
and meters in correspondence with the queues Q r In each queue Qj, packet tags each comprising a flow number "Flow 
s No. (j) B of the received packet, packet length 'PacketLen. (L)', arrival time (t), and a pointer of an address of the buffer 
for storing the packet data per se, are stored in the order of arrival. 

[0088] In the meter corresponding to each queue Qj, the transfer start reference time Oj ., of the head packet of that 
queue Q s and the bandwidth <Dj reserved by that queue Qj are stored. The multi-queue managing means 3-1 is provided 
with a function of managing both the queues Qj in the active state in which at least one untransferred packet (backlog) 

10 is stored and meters corresponding to the queues and is provided with a function of outputting both the tag of the head 
packet of the corresponding queue and the parameter stored in the meter of that queue according to an output request. 
[0089] Further, it is provided with a function of outputting the changed reserved bandwidth (±0) when the active state 
of a queue changes. For example, when a queue Qj changes from the inactive state to the active state, the amount of 
increase (+4>j) of that reserved bandwidth Oj is output to the reserved bandwidth use ratio y computing means 3-2. 

is Conversely, when a queue Qj changes from the active state to the inactive state, the amount of decrease (-Q>$ of that 
reserved bandwidth <Dj is output to the reserved bandwidth use ratio y computing means 3-2. 
[0090] The reserved bandwidth use ratio y computing means 3-2 is provided with both a <J> b computing unit and a y 
computing unit. The ^ computing unit receives as Ms input the changed reserved bandwidth (±0) from the multi-queue 
managing means 3-1 and computes the cumulative value <^ (<t^ = <J>b ± O) of the reserved bandwidths of active queues 

20 (backlog queues). 

[0091] The ycomputing unit divides the cumulative value ^ of the reserved bandwidths of the active queues (backlog 
queues) by the bandwidth R of the entire resources (output link) and computes the ratio y of the reserved bandwidth 
<D b used by the entire active queues (backlog queues) to the bandwidth R. 

[0092] The clustering means 3-3 is provided with both a ft calculating unit and a clustering processing unit C(ft). 
25 The ft calculating unit divides the packet length Lj of the head packet in a queue Qj by the reserved bandwidth Oj of 

the queue Qj (L/0>j) to calculate the scheduled resource occupation time ft of the head packet. 

[0093] The clustering processing unit C(ft) performs the processing for allocating the related packets to the clusters 

C-, to C M in correspondence with ranges to which those values belong based on the values of the scheduled resource 

occupation times ft of the related packets. 
30 [0094] The cluster managing means 3-4 is linked to the inside-cluster selecting means 3-5 and maintains the state 

where the packet having the smallest value Fj among the scheduled transfer finish times Fj of the packets in the cluster 

is selected as the first order packet of that cluster for each cluster Cj (j = 1 to M). 

[0095] Further, a parameter NCP (Next Cluster Pointer) indicating the pointer of the next order packet for maintaining 
the state where the packets in the cluster are sorted in the order from the smallest value of the scheduled transfer 
35 finish time Fj is added to the packets in the cluster. The packets can be linked each other, in the cluster, in the order 
of sorting by the parameter NCP. 

[0096] The inside-cluster selecting means 3-5 has a function of calculating the scheduled transfer finish times Fj (= 
<Xj + ft X 7) of the packets in a cluster by a request from the cluster managing means 3-4 for one of the clusters stored 
in the cluster managing means 3-4, selecting the packet having the smallest that value F { among the scheduled transfer 

40 finish times F } of the packets as the first order packet, and returning this to the cluster managing means 3-4. Further, 
it has afunction of outputting the scheduledtransfer finish time Fj of the first order packet to the output control means 3-7. 
[0097] The inside-cluster selecting means 3-5 can use, as mentioned before, the procedure of selecting the sched- 
uled transfer finish time from a,- and ft, the procedure of sorting, or the procedure of appropriately switching these two 
procedures in accordance with the processing time. 

45 [0098] The inter-cluster selecting means 3-6 calculates and selects the packet having the smallest value of the sched- 
uled transfer finish time Fj (= + ft X y) from among the first order packets in the clusters C, to C M (note, except the 
cluster which was output immediately before this) and outputs that scheduled transfer finish time Fj to the output control 
means 3-7. 

[0099] The output control means 3-7 compares the scheduled transfer finish times Fj input from the inside-cluster 
so selecting means 3-5 and the inter-cluster selecting means 3-6, selects the packet having the smaller value Fj as the 
top priority packet to be transferred, and outputs the packet tag thereof. 

[0100] In order to perform similar processing with the next packet as the head packet for the queue Qj in which the 
tag of the packet transferred is stored, the output control means 3-7 activates the processing of the multi-queue man- 
aging means 3-1, the reserved bandwidth use ratio ycomputing means 3-2, and the clustering means 3-3 for the next 
55 head packet. 

[0101] The functional means of the packet transfer control apparatus of the present invention were explained above. 
Here, an explanation will be made below of the processing operation by referring to the same Fig. 3 while classifying 
the processing operation into packet output processing and packet input processing. Note that, in the figure, the circled 



11 



EP 0 989 770 A1 



numbers correspond to the numbers in parentheses attached to the processing operation in the following explanation 
and represent the link positions among function modules to and from which the information concerning the related 
processing operations are input and output. Further, in the figure, the contents of the information input, output or held 
at the link or function modules are described in blocks a) to h) by attaching lead lines. 

s 

[1] Packet output processing 

[0102] When the two conditions of both the completion of the packet output (10) by the output control means 3-7 
and the output permission (11) of the next packet are satisfied, a request (12) for processing the next packet, as the 
io head packet, in the queue Q to which the packet which was output immediately before this belonged is sent from the 
output control means 3-7 to the multi-queue managing means 3-1 , the reserved bandwidth use ratio 7 computing means 
3-2, and the clustering means 3-3. 

[0103] The multi-queue managing means 3-1 calls up the new head packet if an untransf erred packet is stored in 
the queue Qj, calculates the transfer start reference time a, of the head packet, and stores the calculated value in the 

is meter. Note, if the related queue Qj has become empty, the multi-queue managing means 3-1 does not outputs anything. 
[0104] The transfer start reference time ctj is set as = t when comparing the arrival time 1 of this packet and the 
scheduled transfer finish time Fj 0 of the packet sent immediately before this and t £ Fj 0 , while is set as cc^ = F } 0 if t < Fj 0 . 
[0105] The multi-queue managing means 3-1 outputs (15) the parameters of the packet length Lj and the reserved 
bandwidth Oj to the clustering means 3-3. Further, the multi-queue managing means 3-1 outputs (1 4) the change (±0) 

20 of the available reserved bandwidth caused by the change of the active state of the queue to the reserved bandwidth 
use ratio 7 computing means 3-2. 

[0106] The clustering means 3-3 calculates (2) the scheduled resource occupation time p when the parameters are 
input, performs (3) the clustering for new head packets in accordance with this value p, and outputs the same to the 
cluster managing means 3-4. 

25 [0107] The reserved bandwidth use ratio 7 computing means 3-2 computes (13) the cumulative value O b of the 
reserved bandwidths of the backlog queues. When there is an output request (12) from the output control means 3-7, 
the cumulative value <D b of the reserved bandwidths is output to the 7 computing means, and the 7 computing means 
computes (4) the reserved bandwidth use ration that is, the ratio of the cumulative value of the reserved bandwidths 
with respect to the bandwidth R of the entire resources, and outputs the same to the inside-cluster selecting means 

30 3-5 and the inter-cluster selecting means 3-6. 

[0108] The cluster managing means 3-4 waits for exactly a period for the clustering for the next head packet to be 
completed when the cluster Cj to which the packet selected as the top priority output packet belonged is notified from 
the output control means 3-7 and sends (7) all packets belonging to that cluster Cj to the inside-cluster selecting means 
3-5. The inside-cluster selecting means 3-5 newly selects the first order packet from among these packets and returns 

35 (9) the same to the cluster managing means 3-4. 

[0109] Further, when the next head packet is clustered into another cluster C k , all packets of the other cluster C k are 
similarly sent (7) to the inside-cluster selecting means 3-5. The inside-cluster selecting means 3-5 newly selects the 
first order packet from among these packets and returns (9) the same to the cluster managing means 3-4. 
[0110] The cluster managing means 3-4 outputs (5) the parameters (ctj, ft) of the first order packets in the clusters 

40 other than the cluster Cj finally sent to the inside-cluster selecting means 3-5 to the inter-cluster selecting means 3-6. 
[0111] The inside-cluster selecting means 3-5 calculates the scheduled transfer finish times Fj (= Oj + Pj X 7) of the 
packets by using parameters ctj and Pj of the packets input (7) from the cluster managing means 3-4 and new reserved 
bandwidth use ratio 7, selects the packet for which the scheduled transfer finish time Fj becomes the smallest from 
among these Fj as the first order packet, and outputs (8) this to the output control means 3-7 and, at the same time, 

45 returns (9) this to the cluster managing means. 

[0112] The inter-cluster selecting means 3-6 calculates the scheduled transfer finish times Fj (= + p { X 7) by using 
parameters (o^, ft) of the first order packets of the input clusters and the new reserved bandwidth use ration selects 
the packet having the smallest Fj from among these Fj as a top priority candidate packet, and outputs this top priority 
candidate packet to the output control means 3-7 (6). 

50 [0113] The output control means 3-7 selects and outputs (10) the packet having the smallest value of Fj from among 
the scheduled transfer finish times F s input ((6), (8)) from the inter-cluster selecting means 3-6 and the inside-cluster 
selecting means 3-5 as the top priority transfer packet and, at the same time, requests (1 2) the processing with respect 
to the next output packet. 

55 [2] Packet input processing 

[0114] The multi-queue managing means 3-1 receives (1 ) the packet tags of the packets for performing the transfer 
service and stores the packet tags in the queues Qj corresponding to the flows. At this time, when a queue Qj is a 
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queue ol the active state, the head packet has already existed in this queue, therefore, the input processing is finished 
by just adding the tag of the newly received packet to the queue Qj. 

[0115] When a packet arrives at a queue Q t in an empty state, this packet becomes the head packet, therefore the 
multi-queue managing means 3-1 calculates the transfer start reference time a, of the packet and sets the in the 
5 meter for the queue Q v outputs (1 5) both the packet length Lj thereof and the reserved bandwidth Oj of the queue Qj 
to the clustering means 3-3, and outputs the amount of variation of the reserved use bandwidth (amount of increase 
0>j) to the reserved bandwidth use ratio y computing means 3-2. 

[0116] The clustering means 3-3 calculates the scheduled resource occupation time ft by L/<D { at the p calculating 
unit, clusters the packets according to the value of the scheduled resource occupation time $ t at the clustering process- 

10 ing unit C(pj), and outputs (3) the result to the cluster managing means 3-4. 

[0117] The cluster managing means 3-4, linked with the inside-cluster selecting means 3-5, calculates the scheduled 
transfer finish times F ; for the cluster Cj to which a packet is newly added based on the first order packet in the cluster 
C jt the parameters (a, P) of the newly added packet, and the reserved bandwidth use ratio y and selects and holds the 
packet having the smallest scheduled transfer finish time Fj as the new first order packet. 

is [0118] Figure 4 is an explanatory view of the packet transfer control by a plurality of schedulers according to the 
present invention. In the figure, 4-1 is an input link, 4-2 is a router (classifier), 4-3 is a first multi-queue, 4-4 is a second 
multi-queue, 4-5 is a first sub-scheduler, 4-6 is a second sub-scheduler, and 4-7 is an upper main scheduler thereof. 
[0119] Figure 4 is an explanatory view for packet scheduling in a case where traffic packets of applications for which 
different qualities of service (QoS) are required are mixed such as the transfer of voice packets etc. where a low jitter 

20 is important or the transfer of WWW, FTP, and other data packets where a low delay and maximum transfer capability 
are important. 

[0120] From the input link 4-1, traffic data having different quality of service (QoS) classes such as voice packets 
•Voice", video packets 'Video', and data packets 'Data' are input. The router 4-2 analyzes the header information of 
the input packets to classify these packets into flows to which they belong and stores the packets in the queues cor- 

25 responding to the flows. 

[0121] The queues are comprised as a plurality of multi-queues controlled by different schedulers in accordance with 
the quality of service required by the corresponding fbws of the queues. The first multi-queue 4-3 stores the packets 
for which low jitter is required, for example, voice packets 'voice' and video packets 'Video'. The scheduling is con- 
trolled so that the predetermined output rate is guaranteed by the first sub-scheduler (S1) 4-5. 

30 [0122] The first sub-scheduler (S1 ) 4-5 fixes r t 0 = r t t = <l>j in equation (1 ) mentioned above since it is not necessary 
to reallocate the remaining bandwidth among the queues, calculates the scheduled transfer finish times Fj ^ of the head 
packets in the queues to select the packet having the smallest value F ip1 as the top priority output packet. 
[0123] The first sub-scheduler (S1) 4-5 has a light load of calculation processing since the allocated bandwidth 
is a fixed value, so can perform priority selection processing following at high speed even for processing of a flow 

35 comprised by short packets like voice packets. 

[0124] On the other hand, packets for which a low delay is required like data packets are stored in the second multi- 
queue 4-4. The scheduling is controlled so that the remaining bandwidth is fairly allocated by the weighted fair queueing 
(WFQ) by the second sub-scheduler (S2) 4-6. 

[0125] The weighted fair queueing (WFQ) has the excellent features of the guarantee of the minimum delay of the 
40 packets and the fair reallocation of remaining bandwidth among flows. The second sub-scheduler (52) 4-6 performs 
the clustering mentioned above in the calculation of the scheduled transfer finish times of the head packets in the 
queues, performs the calculation processing at high speed by limiting the head packets to be calculated, and selects 
the packet having the smallest value as the top priority output packet. 

[0126] The upper main scheduler (S) 4-7 links the first and second sub-schedulers (S1, S2), selects the packet 
45 having the smallest scheduled transfer finish time Fj , as the output packet for the output candidate of the packet 
selected from the two (S1 , S2), and outputs this. 

[0127] By such processing, it is possible to guarantee low jitter and guarantee low delay for the flows in accordance 
with the quality of service (QoS) required by the flows and effectively utilizing the output link fairly among flows to the 
highest limit for the package transfer. 

50 [0128] Summarizing the effects of the present invention, as explained above, in the processing of weighted fair 
queueing (WFQ) for selecting and transferring the head packet with the earliest scheduled transfer finish time from 
among the head packets in queues for which the scheduled transfer finish time varies due to the change of the sum 
value of the reserved bandwidths used by the backlog queues, by clustering the head packets with a low dependency 
of changes in the relative order of the scheduled transfer finish times on changes in the sum value of the reserved 

55 bandwidths of the backlog queues, calculating the scheduled transfer finish times with respect to only the first order 
packet in each queue by reusing the output priority order in each cluster calculated in the past as it is, and selecting 
the top priority packet, and thereby the amount of calculation in the weighted fair queueing (WFQ) processing can be 
greatly reduced. 
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[0129] Further, in the calculation of the scheduled transfer finish time, the time axis is the real time axis, therefore 
the selection processing with respect to the packets with the top priority individually selected by a plurality of schedulers 
becomes easy and the selection processings by the plurality of schedulers can be easily linked. 



Claims 

1 . A packet transfer control apparatus comprising: 

w a scheduler for controlling the transfer by weighting the order of transfer for each head packet in the queues, 

the scheduler comprising: 

means for dividing each head packet in the queues into clusters in accordance with the ratio between the 
packet length of the head packet in each queue and the reserved bandwidth of the packet, and 
means for selecting the packet having the earliest scheduled transfer finish time as the top priority transfer 
is packet from among each first order packet inside the clusters. 

2. A packet transfer control apparatus provided with at least one scheduler which controls the transfer by weighting 
the order of transfer for the head packets in the queues in the order from the packet having the earliest scheduled 
transfer finish time for packets stored in a plurality of queues to which are allocated reserved bandwidths individually 

20 determined in advance and, at the same time, allocating the resources (bandwidths) to all of the active queues in 

which packets to be transferred are stored, 
wherein the scheduler is provided with: 

a clustering means for dividing the head packets in the queues into clusters in accordance with the ratio 
25 between the packet length of the head packet in each queue and the reserved bandwidth (hereinafter referred 

to as the scheduled resource occupation time), 

a cluster managing means for holding the packets having the earliest scheduled transfer finish times among 
the packets in a cluster for all clusters as the first order packets, 

an inside-cluster selecting means for selecting the packet having the earliest scheduled transfer finish time 
30 for the cluster as the first order packet when a packet is deleted or added inside the cluster, and 

an inter-cluster selecting means for selecting the packet having the earliest scheduled transfer finish time as 
the top priority transfer packet from among the first order packets inside the clusters. 

3. A packet transfer control apparatus as set forth in claim 2, wherein: 

35 

said scheduler is provided with a reserved bandwidth use ratio computing means for computing the ratio (here- 
inafter referred to as the reserved bandwidth use ratio) of a cumulative value of the reserved bandwidths for 
the active queues with respect to the resources (bandwidths) and 

said inter-cluster selecting means is structured to calculate the scheduled transfer finish time based on the 
40 transfer start reference time of each first order packet, the scheduled resource occupation time, and the re- 

served bandwidth use ratio. 

4. A packet transfer control apparatus as set forth in claim 2 or 3, wherein: 

45 said inside-cluster selecting means is structured to sort the packets in a cluster in the order from the earliest 

scheduled transfer finish time and 

said cluster managing means may be constituted so as to hold the packets sorted in the order from the earliest 
scheduled transfer finish time for every cluster. 

so 5. A packet transfer control apparatus as set forth in claim 2 or 3, wherein: 

said inside-cluster selecting means is structured to perform the selection by switching between processing for 
selecting only the first order packet having the earliest scheduled transfer finish time from the packets in a 
cluster and processing for sorting them in the order from the one having the earliest scheduled transfer finish 
55 time, 

perform the processing for sorting in the order from the one having the earliest scheduled transfer finish time 

when the transfer time of a packet in transmission is more than a predetermined time, and 

perform the processing for selecting only the first order packet having the earliest scheduled transfer finish 
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time when the transfer time of the packet in transmission is not more than a predetermined time. 

6. A packet transfer control apparatus as set forth in any one of claims 2, 3, and 5 t wherein the inside-cluster selecting 
means is constituted so as to select the packet having the earliest scheduled transfer finish time based on only 

5 said transfer start reference time of each packet in the cluster or said transfer start reference time and said sched- 

uled resource occupation time. 

7. A packet transfer control apparatus as set forth in claim 4 or 5, wherein said inside-cluster selecting means is 
constituted so as to sort the packets in the cluster in the order from the one having the earliest scheduled transfer 

10 finish time based on only said transfer start reference time of each packet in the cluster or said transfer start 

reference time and said scheduled resource occupation time. 

8. A packet transfer control apparatus as set forth in any one of claims 2 to 7, wherein said clustering means is 
constituted so as to divide the head packets in the queues into clusters by discriminating the type of the application 

75 packet having traffic characteristics for which the scheduled resource occupation time is specified. 

9. A packet transfer control apparatus as set forth in any one of claims 2 to 8, wherein: 

provision is made of a plurality of schedulers which perform different processing for allocation of the resources 
20 of the entire transmission system for packets stored in multi-queues differing for all clusters in accordance 

with the class of the quality of service, 

each scheduler is provided with a means for selecting the packet having the earliest scheduled transfer finish 
time from among the head packets for every multi-queue, and 

provision is made of a scheduler for selecting the packet having the earliest scheduled transfer finish time 
25 from among packets selected by the schedulers corresponding to the multi-queues. 

10. A packet transfer control apparatus as set forth in claim 9, wherein at least one scheduler among said schedulers 
corresponding to the multi-queues is provided with a means for allocating bandwidths of a fixed rate to queues in 
the corresponding multi-queues and selecting the packet having the earliest scheduled transfer finish time. 

30 

11. A packet transfer scheduling method which transfers packets by weighting the order of transfer for head packets 
of queues in the order from the packet having the earliest scheduled transfer finish time for packets stored in a 
plurality of queues to which are allocated at least reserved bandwidths individually determined in advance and, at 
the same time, allocating resources (bandwidths) to all of the active queues in which packets to be transferred are 

35 stored, comprising: 

a step of dividing the head packets of the queues into clusters in accordance with scheduled resource occu- 
pation times of the head packets in the queues, 

a step of selecting the packet having the earliest scheduled transfer finish time for a cluster as a first order 
40 packet when a packet is deleted or added to packets in the cluster and holding this selected packet, and 

a step of selecting the packet having the earliest scheduled transfer finish time from among the first order 
packets in the clusters as a top priority transfer packet. 

12. A packet transfer scheduling method as set forth in claim 11, further comprising a step of computing a reserved 
45 bandwidth use ratio of active queues and a step of calculating a scheduled transfer finish time during the selection 

among clusters based on the transfer start reference time of the first order packet, the scheduled resource occu- 
pation time, and the reserved bandwidth use ratio. 

1 3. A packet transfer scheduling method as set forth in claim 1 1 or 1 2, further comprising, when achieving the selection 
50 in the clusters, a step of sorting the packets in a cluster in the order from the earliest scheduled transfer finish time 

and holding the sorted packets. 

1 4. A packet transfer scheduling method as set forth in claim 1 1 or 1 2, further comprising, when achieving the selection 
in the clusters, a step of performing processing for sorting in the order from the packet having the earliest scheduled 

55 transfer finish time when the transfer time of a packet in transmission is more than a predetermined time and 

performing processing for selecting only the first order packet having the earliest scheduled transfer finish time 
when the transfer time of the packet in transmission is not more than a predetermined time. 
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15. A packet transfer scheduling method as set forth in claim 11, 12, or 14, further comprising, when achieving the 
selection in the clusters, a step of selecting the packet having the earliest scheduled transfer finish time based on 
only the transfer start reference time of each packet in the cluster or the transfer start reference time and the 
scheduled resource occupation time. 

5 

16. A packet transfer scheduling method as set forth in claim 1 3 or 14, further comprising, when achieving the selection 
in the clusters, a step of sorting the packets in the cluster in the order from the one having the earliest scheduled 
transfer finish time based on only the transfer start reference time of each packet in the cluster or the transfer start 
reference time and the scheduled resource occupation time. 

10 

17. A packet transfer scheduling method as set forth in any one of claims 11 to 16, further comprising, when achieving 
the selection in the clusters, a step of dividing the head packets of queues into clusters by discriminating the type 
of the application packet having traffic characteristics for which the scheduled resource occupation time is specified. 

75 
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